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In order to enlarge the sanple of educable aentally 
retarded children used in the evaluation of ledia at the Conputer 
Based Project, a number of classes were transported by bus to the 
project site. The investigator questioned whether observed 
differences in manifest anxiety might affect responses to the media 
being evaluated. A study was made to ascertain whether or not there 
were differences between transported and nontransported students with 
respect to; (1) performance in answering criterion item questions, 
and (2) time required to administer the items. The same film was 
shown to both groups ; at its conclusion, a posttest consisting of 10 
or more items related to the film seen and another not seen was 
administered. The posttest performance (items correct and time 
required for administration) of transported students was compared 
with that of a similar age-grade class which was not transported. The 
results indicated that there is no significant difference between the 
posttest performance of the bussed and the nonbussed groups. The time 
required for administering the test was longer for the bussed groups. 
(Author) 
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AN INVESTIGATION OF THE EFFECTS OF TRANSPORTING 
EMH STUDENTS TO A MEDIA EVALUATION SITE 

ABSTRACT 

In order to enlarge the sample of educable mentally retarded 
children used in the evaluation of media at the Computer Based Project 
(Syracuse, N. Y.), a number of classes were transported by bus to the 
Project site, The investigator questioned whether observed differen- 
ces in manifest anxiety and attention between transported and non- 
transported students might affect responses to the media being evalu- 
ated. The purpose of this study was to ascertain whether or not there 
were differences between transported and non-transported students with 
respect to: (1) performance in answering criterion Item questions, and 
(2) time required to administer the items . The same film was shown to 
both groups i at its conclusion, a posttest conglatlng of 10 or more 
items relfeted to the film seen and another not seen was administered, 
The posttest performance (items correct and time required for adminis- 
tration) of transported students was compared with that of a simila» 
age-grade class vhich was not transported. The results indicated that 
there is no significant difference between the posttest performance of 
the bussed and the non-bussed groups. The time required for administer- 
ing the test for the groups indicated that it may take longer for • 
bussed students to complete a test. Significant differences were found 
between primary and intermediate groups, primary groups receiving . 
'■^''•lo\ifir1'c&r 

. .::;:SIi|*ti9H»LTNU(tjUte*!(-i3 

..£s»M'g-si;;i'6utW>M*Se 

iKafMU''? c>Sttii* WtSS^ PRO ^-aHi/iVsWSa:! 

; the P£«^ 



SPECIAL REPORT No. 731 
COMPUTER-BASED PROJECT for the EVALUATION 
of MEDIA for the HANDICAPPED 

T I t I 6 ! ^ INVESTIGATION OP THE EFFECTS OF TRANSPORTING EMH STUDENTS TO A 
MEDIA EVALUATION SITE 

BACKGROUND ^ 



The Computer Based Project for the Evaluation of Media for the Handicapped, 
based on contract j^OEC-9-423617-4357 (616) between the Syracuse (N. YO Clty-^chool 
District and the Media Services and Captioned Films Branch, Bureau of Education 
for the Handicapped (United States Office of Education) for the five year period 
July Ir 1969 through June 30/ 1974, The major goal is to improve the instruction 
of handicapped children through the development and use of an evaluation system 
to measure the instructional effectiveness of films and other materials with 
educable mentally handicapped (EMH) children, in-service training and media support 
for special teachers, and studios related to the evaluation process and the 
populations used. 

The Project has concentrated on the 600 films and 200 fiJjnstrips from the 
Media Services and Captioned Films (BEII — USOE) depository; however, specific 
packages from Project life; various elementary math curricula, and selected 
programs frbrn Children's TV Wor)cfihop have also been evaluated. The ovaluation 
model used requires that: 1) objectives of materials be specified and writteni 

2) instruments be constructed to test and measure effectiveness; andi 3) children 
be the major sources of evaluation information • A nuitiber of instrumonts and 
methodologies ar^ employed in the gathering of cognitive and affective data from 
900 EMH children and 80 special teachers to make the ef feotlveness deci8ion$ . 
Over half of the EMH population can neither read or write; therefore, a unique 
Student Response System (SRS) is employed, consisting of a twenty station G»E*-^ 
1000 SRS which can be operated in a group or individual recording mode and is 
connected to a remote computer system. The computer capabilities consist of 
remote telephone connections to the Rome (N.Y.) Air Development Command, the 
Honeywell time-shared network, and the Schenectady (N*Y.)GE Research and 
Development Center; and batch mode capabilities of the Syracuse City Schools, 
Syracuse University, and various cotrimercial sources. 

In- service and nedia support activities provide bn-tlm-job training for 
.teachers, teacher aides, equipment, and materials to the special teachers in 
the city schools. The research activities have centered around Invostigations 
and special piroblsms related to the development of the evaluation model. The 
four major areas considered arer 1) testing effects, 2) captioning effects, 

3) special student characteristics; and, 4) evaluation procedures validation. 



Documentation of the major activities appear in the five annual reports 
lihd Jthia 6pO>yalua prepared on ma^rials u^ed. Staff members v/oi?e pncpuraged 
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AN INVESTIGATION OF THE EFFECTS OF TRANSPORTING 
EHR STUDENTS TO A MEDIA EVALUATION SITE,. 

The Computer Based Project for the Evaluation of Media for the Handicapped 
(CBP| 1969) Is detcrmtning the Instructional effectiveness of selected captioned 
fllrfis for educationally mentally handicapped (EMR) children. In an effort to 
broaden the sample of children used In the evaluation process » a number of i 
classrooms are transported to an established center In an fnter-cfty school of 
a large city school system. At this center the children view a given film and 
respond to cognitive and affective Items Intended to measure effectiveness. 
PROBLEM 

Over a period of time some observations have been made of apparent differ- 
ences In anxlety» attention, and to a lesser degree, In attitude, of those 
students who have been transported from other schools Into the Computer Based 
Project (CBPj 1969) site for media evaluation purposes and those not transported* 
It would seem that the additional commotion brought upon by the bussing process 
tiiay heighten the exhibited levels of activity for many of the bussed students 
to a greater extent than those who are residents (not bussed In) ac the school 
site. This heightened behavior m^y reduce the effectiveness of the educational 

• » Mr 'i 

process afforded at CBP as a discrepancy In the reported performance on the 
criterion test and the length of time required to administer the various pre 

and post test Items, consisting of several cognitive and affective questions 
administered to the bussed and resident populations. 



ASSUMPTIONS 

Several general assumptions are made In conducting thts study > specif leal ty: 
1. AH the children In EMR classrooms perform as though they are retarded* 
2* Classrooms with the same age^grade designation are representative 

samples of that strata of the EMR population. 
3. The conditions for viewing films at the center are standard, 
^i; The matrix sample of media Items used as a posttest Is a valid 

measure of cognitive achievement. 
S» The effects of different films are the same. 

$• The effects of a film are considered standard for a given level of child. 
LIMITATIONS 

Because this study was done using data regularly generated for the 
evaluation process rather than being organized as an independent study the 
following limitations are recognized. 

1 . The actual Items making up the post test may vary from presentation 
to presentation because of the nature of making Item sets uses an 
Independent sorting process which results In different Items being 
placed together for a given presentation. 

2. Classrooms do not have equal numbers of members. 

DESIGN 

The standard evaluation process at CBP are followed: The classrooms 
arrive at their regular scheduled time and view a f I Im scheduled for evalu- 
ation without modification except for Increasing the number of classrooms 
viewing , a given f 1 Im. iMfM^^^^ 
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SAHPLE 

The classrooms scheduled Into the on-going regulaf evaluation prgccasc^ 
were classified as transported (Group I) or resident (Group II). A sample 
consisting of six classes In each group who had viewed the same film and 
taken the posttest consisting of at least four of the Items presented to all 
groups. A further subgrouping is made pf three classrooms, each at the 
Primory and Intermediate levels as deter'mlned by the ago-grade classff feat Ion 
of the School District's Special Education Department* 

INST RUMENT AT rO M ' 

The criterion Instrument will be the regularly scheduled posttest of 
10 cognitive Items. Five are prepared to measure achievement for the film 
''seen" and five having to do with a f IJjn that was not seen (''unseen'O « 
Four Items deal Ing with liking the film are added to the above posttest set 
and administered Immediately following the viewing of the film. The criterion 
test given each group varies slightly from class to class because of the 
matrix sampling procedure employed to select Items. An InstruiTient of responses 
will be statistically constructed by computing a proportion correct for each 
class on the posttest cognitive Items and using this as a score for the group. 

TREATMENT 

Each classroom was shown a film under the regular evaluation process 
and scheduled for viewing a fl Im In the evaluation center's student response 
room, and following the film, administered the posttest. A stop watch was 

^^^^^^^^^^^^^^^^^^^^^ - s the first Item was asked and stopped after the last Item had been 

responded to by the class. The total elapsed time for the posttest adminis- 
tration was so recorded for each classroom. 
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DATA ANALYSIS 

The sample population was organized Into a 2 x 3 design with two Independ* 
erit measures cognitive score and time to administer the Items, Two 
categories of academic level ore recognized ao primary and tntermedlate with 
three classrooms representing each level with one classroom matched on one of 
three films and the criteria of being transported and not being transported 
as shown In Figure I . ^ 




Two separate data col lections were done for Primary and for Intermediate 
to allow for the different films used In each grade level because of the 
schedule arid classification of appropriateness In the evaluation process. 

The data was processed through the Project *s student response media 
evaluation system and reduced by a computer program giving crfterton test 
Item scores for each class on each Item In the various questions administered 
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Irnmcdtately after the students have viewed the media. Because the number 
of Items answered varied between classrooms a mean Item score was computed 
by dividing the number of Items answered Into the mean criterion score 
obtained for the class thus yielding a value comparable across classroomsi 
as shown In Table f below. 



TABLE ONE: SUMfWRY DATA FOR PRIMARY AND INTERMEDIATE GROUPS 
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The ^'Number of Items^* column Indicates the Items related to the cognitive 
Information from the fl Im seen. The "Total Questions^' column Includes all 
Items; consisting of cognitive Items on the film seen (generally 5 to a set) , 
cognitive Items on a film not seen (generally 5 to a set) , and affective I tems 
dealing with the class's attitude toward the film seen (generally h Items) 



for a total maximum of l** Items. The variance from this maximum Is caused 
by differences In the makeup of the Item set; I.e., number of Items available 
paired or not paired with another film and administering them, t.e«, the 
length of time available for seeing the film and asking questions, presence 
or absence of Intervening activities (fire drills, bus schedules, special 
school programs, etc.). 

Summary Table One Indicates the reported data for the primary and Inter- 
mediate levels, bussed and resident populations. The time required for the 
system operator to administer the test Items Is shown tn terms of the number 
of questions divided by total time. 

DISCUSSION 

The first null hypothesis that V*There Is no significant (p " .05) 
difference between bussed and resident groups on number correct and time to 
administer,-' was te$ted using the Hann^Whitney U test. This test Is considered 
to be one of the more powerful of the nonparametric measures for determining 
whether or not two independent groups have been drawn from the same population 
(Slegel, 1956; Roscoe, 1971) • This test assumes that the ordering of the 
scores will occur In a systematic way such as 

0 R B R B R B R B R 

i or S 
R B R B 

and detects a significant change from this arrangement. The actual confiputa- 
tlons are summarized below In Table 11 for Primary level and In Table III 
for Ifitermedlate le^^^WMMM^^^ 



TASLe TWO 
CRITERION SCORE 



MAim-WMlTUEY U TEST FOR PRIMARY LEVEL, BUSSED AtJO RESIDENT STUDENTS 



SCORES .36 ' .37 .^>5 M .50 .62 

GROUP Bus Res. Bus Res. Res. Bus 



Computation of U: n. - 3; n, - 3; P - .5 (Table J ) 

1 + 2 + 2 •» Uf - 5 
1 + 3 " Ug - k 

A tabled value of U - 4 with and N2 » 3 (Table J, p. 271 , Selgol , 1956) 
results In a p - .5 v/hlch Is greater than our hypothesised p - .05. 
Therefore, we find that no significant difference exists between primary 
groups on the response to the media test Items (mean criterion test score). 

TABLE III :• : : : 

MANN-WHITNEY U TEST FOR I NTERIdEDI ATE LEVEL, BUSSED AND RESIDENT STUDENTS 



SCORES .43 .50 .75 .76 .89 42 

GROUP Bus Res. Bus Bus Res. ^itJus 



Computation of Us n^ - 3; n2 " 3; p .5 (Table J) 



1 + 1 + 2 - U^ 'k 

2 + 3 - Ujj « 5 
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The tabled probablHty for U ■ 4 when H\ and N2 3 Is P «5 

Here atsoi no significant difference exists between secondary groups 
on the response to the mean criterion test Items* P « .5 > P •^S. The 
analysis of the time to administer the Items will follow later. 

A second hypothesis was the ^ The re are no differences between the scores 
between primary and Intermediate level/' A Mann-WhItney U test Is summarized 
In Table IV below: 

TABLE IV 

MAMN-WHITNEY U TEST FOR DIFFERENCES BETWEEM LEVELS 





S CORPS 
GROUP 


.36 .375 .'»3 
P P I 


M 
P 


P 


.50 .50 .62 .75 .76 .89 .92 
P I P I 1 1 1 



Computation of U: .M^ 6 « 6 p « .02 

U) « 2 + 5 + 6 + 6 + 6 + 6 » 31 
Up * 1 + 1 +1 + 2 » 5* 



The probabi llty of obtaining a U * 5 1s p « .02 which Is less than our 
p « .05; therefore, there Is a significant difference between the levels on 
the percent correct on cognitive Items. In Table l» It can be noted that 
both the bussed and resident mean I tern correct values for Intermediate are 
roughly 50% more than those for primary, 

TO ADMINISTER CRI Till 
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TABLE V 

U TEST FOR DIFFERENCES IN TIME PER ITEM 
RANK BY MEAM T(H£ PER ITEM 



P 

SCORE .52 .55 .71 .86 .95 1 .0 1 .02 1 .1 I . I9 1.23 1-75 U Value 



GROUP 

by 

Situation R R R B R B 8 R B 8 B R 10 .12 

Primary X X X X X X 0 .05 

by Level I P P I ) | I P p P P I 14 .29 
Rank 

by Score k 11 6.5 1 2 10 3 8 12 9 5 6.5 rho«>-20 



No significant differences were found for Intermediate 1 eve T sub-group 
when ctassed as resident or bussed; however^ there was a significant difference 
for primary level with bussed subjects (Kean time « 1 J?) talcing longer for the 
testing situation than resident students (Mean time « .96) • No significant dif- 
ferences were present between levels when control for transportation was not 
made; however > a perusal of the data suggests a tendency for primary children 
to take longer than Intermediate as Indicated by four Intermediate mean times 
being KO or less whereas only two of the primary times were In this category. 
The extremely long time (1.75) for the f?6 Intermediate group suggests something 
may have Interfered wt th the test such as a behavior problem, teacher Interruptlonj 
etc. If the U test Is recomputed omitting this group» a U » 8 Is obtained* The 
probability of this value Is p a .12 for N| «« 5 and Ng * 6 which still Indicates 
a non-slgnlf leant difference In sub-group level (at the p « •OS level)* 

The rank-order correlation between the obtained scores and the time to 
od.n|nl$ter the i tern 1 3 ?jiow'n In T<^h Ve V the result fng^^^^^^r^^^ ^ -tM ?w99??tj'^9 



esschtlalty no correlation between the percent correct and the time to take 
the test. The fact that rho U negative may even suggest that tlie longer 
time taken for the test, tends to lower the score* 

The results Indicate no significant differences on cognitive Items 
between groups bussed to the Project and those not being transported to 
the Project . The s I gnl f I cant d I f f e rences between 1 eve I s on percent correct 
were consistent and as are expected; I .e./ Intermediate children are ex* 
pected to have higher correct scores than primary children* There does seem 
to be a difference In the time to administer the test Items to primary 
chl Idren, with those being transported taking the longest time. A small 
negative correlation between the percent correct and the time to take the 
Item was found. 

CONCLUSIONS 

These findings tend to suggest that transport Ingchl 1 dren to the Project 
site has had no appreciable effect on thel r response and cognitive Items as 
long as the extra time to administer the Items to transported primary 
chl Idren can be allowed .It Is further Implied that transporting the chil- 
dren seems not to have affected thel r cognitive responses. 
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